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ISSUES IN INDEXING FUNCTIONAL LITERACY 

Thomas G. Sticht 
Applied Behavioral & Cognitive Sciences, Inc. 
San Diego, CA 9.?106 

In a speech of September 18, 1985, Secretary of Education, William J. 

Bennett outlined seven principles that would guide the U.S. Department of 

Education as it developed plans for the future of the National Assessment of 

Educational Progress (NAEP). It is the fourth principle that is the subject of the 

present paper. That principle states: 

NAEP should develop an index of "functional literacy" that Is consistent 
overtime and applicable to the adult population as well as to children of 
school age. NAEP should then employ a fixed schedule (once every decade, 
perhaps, like the Cencus) by which it uses this index to assess literacy and 
illiteracy in the nation as a whole. 

At the end of his speech. Secretary Bennett states that, "Fundamentally, we 
all use assessment-type data for diagnostic purposes, so that we can know how we 
are doing, where we are succeeding up to our aspirations and where we are falling 
short, in order to strengthen our ability to provide every child with an opportunity to 
achieve educational excellence." 

This paper explores some of the issues involved in developing an index of 
functional literacy that can serve diagnostic purposes like those addressed by 
Secretary Bennett. Previous discussions cf the concept of functional literacy and 
its assessment have been given by Northcutt, et al (1973), Bormuth (1975), Rosnick 
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and Resnick (1 977), Kirsch and Guthrie (1 977/7Q), Fisher (1 978), Hunter and 
Harmon (1979), Levine (1982), Cervero (1985), Stedman and Kaestle (1986), 
Valentine (1 986), and Kirsch and Jungeblut (1986). While these and other papers 
have been reviewed in preparing to write this report, limitations of time do not 
permit me to comment on each of the earlier works, nor to incorporate all of the 
important points made by the authors of these earlier works. They are noted here 
so that readers, of this papor may seek them out for further study. 

There are a number of important issues not considered in this paper that are 
involved in any national assessment, such as the representativeness of the sample 
used to extrapolate findings to the population, the adequacy of the data processing, 
analysis, and reporting, and the general limitations of psychometric methods for 
scaling and indexing any human "trait" or "ability," including basic measurement 
theory, unit of analysis and other scaling problems, generalizability questions and 
so forth. All of these are important matters for educational achievement, but they 
are not unique to literacy assessment and so they are not dealt with here. 

Here I will limit discussion to what 1 think are basic issues in the assessment 
and indexing of functional literacy. I will not discuss issues that I think are so well 
understood today that they do not need further discussion. This includes problems 
in using years of education to indicate levels of literacy. It is so well recognized 
now that even possession of a high school diploma does not guarantee a high 
level of achievement in literacy that it is not worth a protracted discussion here. 

Nor will I comment on the problems of using with adults standardized reading 



tests normed on grade school children and scored in reading grade ievel units. 
Elsev/here I have shown that, apart from the poor psychometric properties of 
reading grade leval scales (e.g., lack of interval scaling), aduita who scok^ at, say, 
the 5th grade level on a standardized reading test may not be as competent as 
typical 5th grade students in other literacy tasks. Hence, the basic meaning of a 
reading grade level score, that is, that the examinee who gets a given score has 
the skill of the norming group at that score does not necessarily hold for adults 
taking tests normed on school children (Sticht, 1982: see also Stedman and 
Kaestle, 1986, Appendix C). 

Another issue that I will not dwell or concet-ns the sociopolitical differences 
among individuals and groups that prevent unanimity regarding the development 
and utilization of any type of test or index of functional literacy. There are some 
who reject the concept of national human resources management policies and any 
attempts by the federal (or any other) level of government to "diagnooe" 
educational problems and to direct, guide, or even inform efforts to see if "we" are 
reaching "our" aspirations. For Instance, Hunter and Harmon (1979) have argued 
that each person ought to select his or her own literacy goals. They defined 
functional literacy as ". . . the possession of skills perceived as necessan/ by 
particular p ersons and groups to fulfill their own self-determined objectives ... the 
ability to read and write adequately to satisfy the requirements they set for 
themselves as being Ihportant for their own lives ... (p. 7)" (underlining for 
emphasis is in original text). 
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But even Hunter and Harmon must have recognized the limitations to self- 
determination of literacy goals by individuals when, in the very next phrase, they 
state that functional literates should possess ". . . the ability to deal positively with 
demands made on them by society ... (p. 7.)" Since society and not the individual 
determines societal demands for literacy, self-determination of literacy goals is not 
a feasible concept in contemporary society. We have recognized this by passing 
compulsory education laws in all states for children. Children do not get to self- 
determine when, what, how,or why they learn literacy skills and the knowledge it 
takes to make the literacy skills worthwhile. A question that has not been debated 
is whether or not illiterate or functionally illiterate adults ought to face compulsory 
education. But this is a hornet's nest that I will not disturb here. 

THE NATURE OF LITERACY 

There are two major issues that must be dealt with in indexing functional 
literacy. One of these concerns the nature of literacy; just what do we mean by that 
word? The second major issue relates to Hunter and Harmon's assertion that 
functional literates should possess the ability to deal positively with demands made 
on them by society. The question is, what are the demands for literacy that society 
makes, and on whom? 

What is Literacy? The recent paper by Stedman and Kaestle (1986) 
illustrates the problems of understanding what is meant by the word "literacy." 
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Several quotes make the main point: 

"Nevertheless, of the five tests, the Adult Performance Level was 
particularly subject to the problem of bias because it was designed to test 
knowledge as well as skills" (p. 38). 

(recj^ifding another test) "Bias was probably less a problem since these tests 
were focused on functional skills rather than knowledge. Tests that involve 
telephone dialing instructions, application forms, and everyday reading such 
as train schedules, store coupons, and report cards are less prone to cultural 
prejudice." (pp. 39-40) 

(in critiquing Donald Fisher's information processing model and analysis of 
the errors of people on ETS's Adult Functional Reading Study) "his model 
describes a dynamic information processing system that cannot be clearly 
distinguished from functional literacy." (p. 42) "But college-educated 
individuals, having had extensive test experience, likely recognized most test 
construction errors and realized what the tester "had in mind." The errors they 
made on the test were more likely to have been real functional errors." (p. 43) 

Here, then, is a critical issue: what is the relationship of knowledge to literacy 
skill? Stedman ana Kaestle seem to believe that there are "skills" that do not 
require knowledge for their execution. According to them, testing "knowledge" 
introduces opportunities for "bias" that are not there or are at least greatly reduced 
if testing is restricted to "skills." Somehow, for them, reading telephone dialing 
instructions tests "functional skills rather than knowledge." Yet, how one could read 
and comprehend telephone dialing instructions without possessing knowledge of 
the meaning of such words as "telephone," "dialing," and so forth is unclear to me. 

There is a wide-spread belief in knowledge-independent literacy skills that 
make the notion of "functional literacy" compelling to some. This idea is captured 
by educators in the saying that "First the child learns to read, and then he reads to 
learn." Similarly, the idea that "literacy is the key tool for acquiring knowledge" 
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leaves the impression that literacy is something one first gets and then uses to 
acquire knowledge. 

In the NAEP, the conceptual separation of literacy from knowledge is clearly 
indicated by the existence of tests of reading and writing skills, and separate tests 
for "content," such as mathematics, science, etc. Of course, all of the latter 
"content" tests are given in written language that require reading "skill," just like the 
reading tests do. And if one studies the reading "skill" tests, one finds all sorts of 
specific "knowledge" called for, such as knowing the meaning of the vocabulary 
word "horsepower." 

I will not attempt to resolve the knowledge/process issue here, but I will note 
that it is important to understand the role of knowledge as tlie key tool by means of 
which literacy is acquired. Developmentally, children first acquire knowledge, 
including knowledge of the oral language. They then draw on their knowledge of 
the world and the oral language to learn to read and write. Then they use their new 
literacy abilities to acquire new knowledge, including content knowledge and 
knowledge of a wide range of literacy tools such as graphs, flow charts, procedural 
charts, lists, etc. that do not exist in the spoken language. Assessment approaches 
might be developed that take account of the fact that knowledge can be assessed 
without imposing the requirements for reading, by use of oral language and 
pictures (including TV). Then, if one is interested in the extent to which graphic 
tools of thought and communication can be used to represent knowledge already 
known to be possessed,or to acquire knowledge known to be not possessed, 
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appropriate tests could be developed. By establishing in a literacy-free manner 
what knowledge people have, and then assessing their ability to communicate that 
knowledge using graphic symbol systems, one avoids the issue of "bias" as 
expressed by Stedman and Kaestle, above. 

Functional Literacy. The interrelationship of knowledge and skill in our 
thinking about literacy shows itself when we speak of "civics literacy," 
"technological literacy," "science literacy," "computer literacy," etc. as though 
literacy has to do with a body of knowledge. By this way of thinking, one could be a 
science or computer iliiterate, while being technologically, historically, civically, 
culturally, etc. literate. 

When not thought of as a level of skill, such as the 5th grade level discussed 
above, "functional literacy" is frequently thought of like the various content domain 
literacies except that it has to do with different domains. For instance, in the APL 
study, functional literacy was assessed in the knowledge domains of consumer 
economics, occupational knowledge, etc. In the most recent NAEP for young 
adults, the domains were defined not in terms of content, but rather in terms of 
types of materials: prose, documents, and quantitative. In the latter case, content 
knowledge in mathematics was also involved in defining the domain. Within each 
of these three domains, then, people were not tested for possession of the 
semantic content of the materials, but rather in terms of the types of information 
processing involved. 

Functional literacy is n"«ost usually thought of as the domain of literacy in the 



"real world" outside of the school or academic setting. The concern for functional 
literacy is frequently expressed in terms of whether or not the schools are 
producing "products" who can meet the literacy demands of the "real world" of adult 
responsibilities, such as family, community, work, and continued learning. The 
major issue involved in developing an index of functional literacy is that, as a 
society, we do not have general agreement upon what knowledge everyone 
should possess, and different "real world" tasks demand knowledge, not simply 
some little understood set of content free "skills." 

Deciding on just what the knowledge that all of us should possess raises 
another problem. Who is to do the deciding about what is worth knowing? Many 
shudder at the thought that the federal government, or its contracted agents, might 
make this determination. Some think that private test development organizations 
have made this determination in constructing such important gate-keeping 
instruments as the Scholastic Aptitude Test or the high school equivalency exam. 
Still others think that business and industry have too much say about what people 
should be expected to know so that the United States can compete in the changing 
international marketplace. And the knowledge/skill issue surfaces when business 
and industry state that if the schools will simply teach the "basic skills," they 
(business and industry) will teach the knowledge people need for work. 

Sometimes the conception of functional literacy centers not on the 
knowledge, skill, or information processing aspects of literacy, but rather on the 
instrumental value of literacy. Some regard that literacy as functional that leads to 
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the empowerment of the impoverished, disenfranchised people in our society. Yet 
others regard literacy as functional if it leads to a better job, or improved parenting, 
etc. Usually, however, academic literacy that leads to satisfactory progression 
through the school system and into higher education is not considered as 
functional literacy, even though the instrumental value of education is readily 
apparent in our society. One might argue that if one is in tlie school system, then 
academic literacy is functional literacy. But if one is an out-of-school adult who is 
not pursuing an academic credential, then academic literacy is not functional 
literacy. In short, the person's context determines what literacy is functional and 
what is academic. The latter may be regarded as "nice to know" but not "need to 
know" for adults. 

Confounding this issue of "academic" versus "functional" literacy is the 
empirical fact that if one repeatedly scores h'gh on any number of standardized, 
academic tests of reading, writing, or arithmetic, one is much more likely to be able 
to perform a wide range of "real world" litei acy tasks. It is likely this well- 
established finding that leads people to want to talk about literacy in terms of 
grade-school levels, and as content-free skills. A person who does consistently 
score at the twelfth grade level on standardized, nationally normed tests will, in fact, 
be likely to perform a wide range of literacy tasks, both in and out of school. This 
probably reflects the fact that most such tests sample a wide-range of knowledge 
domains and then select items that discriminate among people. People who know 
infrequently occurring vocabulary terms, for instance, are also likely to know more 
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frequently used words. In general, they are likely to have performed a much wider 
number and type of literacy tasks than people who score low on such tests. 

Criticisms of the use of standardized, norm-referenced tests of literacy to 
indicate functional literacy result not so much from thoir findings with high scoring 
individuals or groups as with the results for individuals or groups who do not score 
so well. Then it may be argued tliat such tests do not accurately index how well 
these people can perform "real life" tasks, or that they possess specific literacy 
skills in targeted domains that meet their needs, and these ui nins are not well 
represented on the standardized, norm-referenced tests. 

To Summarize Briefly. The issue of literacy as knowledge, skill, and 
information processing with the written language and other graphic tools of thought 
and communication underlies many of the problems in the assessment of literacy 
and functional literacy. Presently, the NAEP nor any other assessment 
systematically conceptualizes these different aspects of cognition, and uses this 
conceptual understanding to design and develop tests that can reveal the 
contributions of these different facets of ability to the performance of either 
academic or "real world" literacy tasks. 

SOCIETAL DEMANDS FOR LITERACY 

As noted above. Hunter and Harmon state that, while adults should have the 
right to self-determine their needs for literacy, they should also possess the ability 
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to meet the demands of society for literacy. But, as also noted above, it is precisely 
this point, that is, determining the demands of society, that produces the major 
controversies surrounding the assessment of literacy. Some will argue that the 
literacy demands identified are too low, and, therefore, some part of the population 
Is being cheated by not having higher goals or standards set for it. Others will 
argue that literacy demands have been identified that are too high or inappropriate, 
and, therefore, some part of the population is being stigmatized as "functionally 
illiterate" and perhaps wrongfully being denied access to benefits such as 
education and jobs. We have witnessed the latter course of action repeatedly in 
the last decade as lawsuits brought against employers, including the federal 
government, for using "biased" tests for job selection and classification. 

If societal demands for literacy are identified by an analysis of the types of 
"real world" materials and tasks that people will encounter in some contexts, it may 
then be argued by some that this is "domesticating literacy," that people are not 
being taught io go beyond the status quo and to liberate themselves from the 
clutches of contemporary power structures, but rather to acquiesce to and 
therefore, strengthen these structures that promote socioeconomic class 
differences. 

Yet, if societal demands for literacy are not identified through some analysis 
of "real world" tasks and materials, then how can one determine that an individual 
or group possesses the ability to meet societal demands for literacy? And what if 
one of the societal demands identified is performance on the Scholastic Aptitude 
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Test to the level that one can gain admission to any college one desires to enter? 
Or that one can score on the Armed Forces Qualification Test above the minimum 
requirements established for access to military service, with its education, training, 
and medical benefits. Are such universally applicable gate-keeping devices to be 
considered as real-world literacy demands? They certainly determine the 
distribution of considerable benefits in contemporary society. And, all young men 
are subject to military service in times of emergency. 

Eunctional Literacy as Aptitude. When one proposes to use an index of 
functional literacy as a diagnostic tool, so that we might determine if students are 
learning what the schools are teaching, a functional literacy index is considered as 
an index of achievement. However, if the assessment of functional literacy is 
meant to be diagnostic in the sense that it will predict who will or will not be able 
adequately to meet societies' demands for literacy now and in the future, then the 
index serves as an indicator of aptitude for future performance. In this case, what 
one would like to be able to say is that, if a person or group scores at a given level 
on the functional literacy index, then the probability that that person or group will be 
able to satisfactorily perform or learn to perform other literacy tasks is such and 
such. If it is the predictive value of a functional literacy index that is of greatest 
concern, then it is entirely possible that an assessment that focuses on responses 
to background questionnaires and other biographical information may provide a 
better index of functional literacy than a direct test of knowledge and skill. For 
instance, today the Department of Defense finds that whether a person is a high 
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school diploma graduate or not may be a better predictor of satisfactory adjustment 
to and performance In the Armed Forces than are the mental aptitude tests. Used 
in combination, the biographical data about high school graduation and mental test 
scores predict better than either alone. 

Of course, the "aptitude" conception of functional literacy raises the same 
problems that are involved in establishing the predictive validity of any index. This 
surfaces the thorny Issue of determining what should be the criterion or criteria to 
be predicted. Then, the problem of setting cut-off scores on the functional literacy 
index will arise, and so forth with all the problems attendant to selection and 
classification procedures. Yet, despite all these problems. It should be realized that 
any use of a test score or battery or profile of scores that asserts that a person who 
attains a given score or scores possesses "functional literacy" is implicitly 
committing to the aptitude conception of literacy, with all the benefits and 
drawbacks to aptitude assessments. This includes the predictive validity approach 
as the only established empirical method for evaluating the accuracy of the 
literacy-as-aptitude assessment. 

It should be noted that with the use of item response theory (IRT), as in the 
recent assessment of adult literacy (Kirsch & Jungeblut, 1986), pools of items may 
be scaled such that one can estimate the probability that a person with a given 
scaled score can periorm correctly other test items with known IRT scores. 
However, there is no knowledge of how well the person may perform or learn to 
perform tasks requiring literacy that have not been scaled. So while one may feel 
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confident that a person who scores at, say, the 280 level on the new NAEP adult 
literacy scale for prose comprehension has a fairly high probability of being able to 
perform well in reading novels, there is no way of being certain, for instance, that 
the person would work out well as a book reviewer for a newspaper. There is no 
scaled task for the latter in the present pool of scaled items. Furthermore, there is 
more to being a satisfactory book reviewer than merely being able to read and 
comprehend a prose passage. Perhaps what one might feel confident about is that 
if the person did not work out well as a book reviewer, it is not because of his or her 
inability to read and comprehend prose. 

Functional Literacy as Achievement . The major problem with indexing 
I functional literacy as a mark of achievement is that there is a lack of knowledge of 

the experiences and opportunities for learning that people have throughout the 
breadth and scope of the land and over their formative years. It is, therefore, not 
clear if lack of performance on a functional literacy index represents failure to 
achieve given the opportunity to learn what is being tested on the test, or a lack of 
knowledge, information processing, or skill in the latter due to some missed 
educational experience. As discussed earlier, and illustrated with quotes from the 
recent Stedman and Kaestle (1986) study, the distinction between achievement 
testing with or without the opportunity for learning what is being tested lies at the 
heart of many concerns with "bias" in testing. This is a particularly vexing issue 
when coupled with the literacy-as-aptitude use of a functional literacy index. That 
Is because, while it is predictable that people who have achieved pooriy in literacy 
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will i.ave low aptitude for learning or performir.g in situations that demand wel, 
developed literacy ability, and they are in this respect functionally illiterate, it may 
be thought that a people's basic ability for learning, or "intelligence" is being 
insulted if they score poorly on tests for which they have not the opportunity to be 
prepared due to their particular background, and then those test results are used to 
screen people out of opportunities because of "low aptitude." 

The latter point is not vacuous. It so happens today that the Armed Forces 
Qualification Test (AFQT) assesses vocabulary knowledge, paragraph 
comprehension, arithmetic word problem solving, and basic computational skill, 
two of the so-called "three R's," and these scores are used to screen people for 
entry into military service. They are referred to not as measures of literacy and 
mathematics achievement, but rather as measures of aptitude ortrainability. The 
latter term hedges the distinction between achievement and aptitude to avoid the 
types of complaints of "bias" or lack of opportunity discussed above. After all, if the 
AFQT merely measured educational achievement, then the military might be 
charged seriously with the job of educating people to the point where they could 
qualify. On the other hand, if the tests are considered as aptitude tests, then there 
Is enough suspicion that basic learning ability might be involved, and the military 
should not be held accountable for changing something that may have even 
biological bases, or so it could be argued. 

The point of the foregoing is that it is difficult to develop an index of functional 
literacy that can be used diagnostically to say anything with much precision. Of 
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course, if tasks can be selected that most people will accept as being the kinds of 
things that most adults in the U.S. should be able to perform, and it is founa that 
certain groups consistently perform well and others poorly, then, even if it is not 
certain just why that is the case, a focus for further, more diagnostic efforts has 
been achieved. But if this is the goal for NAEP, then it would seem that it has been 
achieved by any number of approaches. For the same findings regarding ethnic, 
minority, sex, and geographical differences in achievement have been obtained for 
the last 75 years of large-scale, national testing ranging from World War I to the 
present. Some problems have almost been diagnosed to death, since it seems 
that with more knowledge there has been less commitment of resources toward 
solving the problems. It is not clear to me how the development of yet another 
index of functional literacy will change this situation. But I hope it does. 
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